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IMAGE ENHANCEMENT EMPLOYING PARTIAL TEMPLATE MATCHING 

Background 

[1] Systems for handling numerous document and data formats are 

moving towards becoming integrated computerized systems where different devices 
5 are tied to one another to provide solutions to customers' needs. A key factor in 
such integrated systems is enabling an original document, chart, or image to be 
scanned, and then presented so that a person viewing the scanned item does not 
perceive any difference in quality relative to the original. Unfortunately, the scanning 
process can introduce noise and errors into the data representing the scanned 
10 image, resulting in a perceptible diminution in the output. Technology is needed for 
enhancing image data haying scanning introduced noise and errors without requiring 
significant, additional components be added to the system. 

[2] Many computerized systems already include some resolution- 

enhancement technology for improving the quality of scanned and printed 

15 documents. Existing resolution-enhancement technologies employ resolution 

enhancement in the form of templates implemented in logic operations or look-up 
tables. The references cited below describe various methods, devices, and systems 
of image or resolution enhancement to enhance printed images and displays using 
components already incorporated into present integrated computer systems. Image 

20 enhancement presently may be used for resolution enhancement, appearance 

tuning, appearance matching, conversion to gray-scale (e.g., halftone de-screening), 
image restoration, pattern detection and segmentation. 

[3] Existing image resolution-enhancement techniques perform a wide 

variety of digital document processing tasks using template-based image 

25 enhancement devices. Enhancement is commonly performed on monochrome and 
color images, as well as binary and continuous tone images. A typical image 
enhancement device includes template operators to enhance the images, which may 
be characterized as an operator or device that transforms one image into another 
image. The image enhancement device is formed of a number of imaging template 

30 operators, often referred to as templates. The templates typically comprise a mask, 
a pattern, and an enhancement pixel. The number of templates in an image 
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enhancement device may vary between a small number of templates to thousands of 
templates. These templates may be, for example, stored in a look-up table and 
implemented using a look-up table formalism. Other equivalent formalisms, such as 
Boolean logic may be employed. 

5 [4] A raster is a one-dimensional array of image data, reflecting a single 

line of data across a single dimension, i.e., the length or the width, of the image. 
Further, each location, or "picture element," in an image may be called a "pixel." In 
an array defining an image in which each item of data has a value, each value 
indicating the properties of a location may be called a pixel value. Each pixel value 
10 is a bit in a binary form of an image, a gray-scale value in a gray-scale form of an 
image, or a set of color-spaced coordinates in a color-coordinate form of an image. 
The binary form, gray-scale form, and color-coordinate form are each arranged 
typically in a two-dimensional array of lines and columns, which defines the image. 

[5] Tung, C, "Piece-Wise Print Image Enhancement for Dot Matrix 

15 Printers," U.S. Pat. No. 4,847,641, Jul. 11, 1989; and U.S. Pat. No. 5,005,139, Apr. 
2, 1991, describe resolution-enhancement methods and devices. Tung describes 
several types of printing errors that occur in printing characters and graphs using 
pixels, such as stepped edges instead of smooth edges, and omitted detail in sharp 
tips and circles (Cols. 7 and 8 of '641 ). Tung describes resolution-enhancement 

20 methods and devices in the prior art operable to smooth printed lines and curves, 
and fill in display errors by enhancing the resolution of pixels. In the prior art, an 
image-resolution-enhancement device uses templates to transform certain pixel 
patterns in a binary image into a corresponding enhanced pixel patterns by matching 
a template to the pixel patterns. Template matching is a binary-matching process 

25 whereby a binary template is applied to a binary image. An image consisting of lines 
in a binary form is sent to a data-buffering module. A binary-pattern-matching 
operation is performed using binary templates that are typically defined as 
possessing ones and zeros. After combining the buffered image data and the 
templates in the binary-pattern-matching operation, an output step substitutes an 

30 enhancement pixel for the target pixel when a 100-percent match is found to create 
the enhanced image. The image-resolution-enhancement process may be 
implemented in various ways, such as a Boolean logic module or a look-up table. In 
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the Boolean logic representation, pixel values are used as variables in the logic 
architecture, such as a logical sum of products. 

[6] Specifically, referring to FIGS. 1 and 2, the image-enhancement device 

observes a pattern of pixels in a window defined by a selected template. FIGS. 1 
5 and 2 illustrate certain aspects of the prior art for resolution-enhancement 
technology. FIG. 1 illustrates a 5 X 5 window comprising a plurality of bits 
representing pixels located about a target pixel, including pixels from the same line 
as the target pixel as well as pixels from at least one line above and at least one line 
below the target pixel (if available). A wide vaYiety of types and sizes of windows are 

10 known. The window used in a particular application depends on the image to be 
analyzed and the particular process to be performed on the image. The 5X5 
window illustrated in FIG. 1 includes five lines (1-5), and five columns (A-E), 
representing a 25-pixel block of binary-valued pixels of an image. One pixel in the 
window is the target pixel, which is typically the center pixel 3C, while the other 

15 pixels in the window are the neighboring pixels. The target pixel and the neighboring 
pixels form a neighborhood. The window is typically moved across the data 
representing an image, advancing from target pixel to target pixel. 

[7] FIGS. 2A-D illustrate conventional resolution-enhancement technology 

being applied to the target pixel 3C in a 5 X 5 window. FIG. 2A illustrates an 

20 example of the values and positions of pixels of a one-bit-per-pixel binary image in 
the 5X5 window. The "1"s represent pixels to be turned-on or displayed, and the 
"0"s represent pixels that are not to be turned-on. The image portion in the window 
of FIG. 2A could represent a two-pixel-wide line rising from left to right, such as a 
forward slash. The pixels in the window of FIG. 2A are inputted into a template- 

25 matching operation as discussed below. 

[8] FIG. 2B illustrates a mask portion of a template that establishes 

whether a particular pixel in a window is a pixel of interest. The mask illustrated in 

FIG. 2B may not include all the pixels in the window as pixels of interest. Rather the 

mask defines a pixel-bit subset of the window that may, but typically does not, 

30 include all the pixel bits in the window. FIG. 2C illustrates a pattern portion of a 

template that defines a "prediction-bit subset" that establishes the value each 

particular pixel of interest must have to be considered a match. That is, for a match 
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to be found, all the pixels of FIG. 2A that are in the "1" positions of FIG. 2B must 
equal the respective pixels in FIG. 2C. FIG. 2D illustrates a match being found in the 
prior art when 100 percent of the pixels of interest (the pixel-bit subset of the window 
as defined by the mask of FIG. 2B) in FIG. 2A each have the required value as 
5 defined by the pattern (the prediction-bit subset) of FIG 2C. 

[9] An image-enhancement device attempts to match the pixels of the 

window with one or more of templates. The template or another portion of the 
image-enhancement device includes an enhancement pixel to be substituted for the 
target pixel 3C when an exact match is found. If an exact match is found by 

10 comparing a pixel subset of the window with a prediction-bit subset, the image- 
enhancement device generates an appropriate output. The output may be an 
enhanced-pixel pattern for the target pixel that enhances the image. The image- 
enhancement device typically continues until all the templates have been applied to 
each target pixel, and all the target pixels have been enhanced (provided template 

15 matches have been found). In the example represented by FIGS. 2A-D, a match is 
found, and an enhancement pixel is substituted for the target pixel 3C to make the 
line look less jagged. Such an enhancement pixel may include a bit of toner added 
to target pixel 3C, or darkening only the upper left quadrant of target pixel 3C to 
smoothen the two-pixel wide line in the window of FIG 2A. When a 100-percent 

20 match is not found, the conventional resolution-enhancement technology does not 
generate an enhancement and the value of the target pixel is not changed or 
enhanced. 

[10] The above described resolution-enhancement technology may not 

accurately provide resolution enhancement for scanned images because noise and 

25 errors in the scanned data often make the required 100-percent match unlikely. 
There is need for a method and apparatus using existing image-resolution- 
enhancement technologies to efficiently enhance digital-image data that may contain 
noise and errors, such as might be created in scanning a document. In view of the 
foregoing, there is a need in the art for a new and improved apparatus and method 

30 for resolution enhancement of images having noise and errors in their data. 



4 



Agilent No. 10021274-1 (2116-11-3) 



Summary 

[11] An aspect of the invention provides a method of image-resolution 

enhancement employing partial template matching. The method includes the steps 
of storing at least a portion of an image, and selecting from the image a window 
5 comprising a plurality of adjacent line segments having pixels, the window including 
a target pixel. The method also includes comparing the pixels of the window with a 
template for a partial match, and responsive to a partial match being found, 
substituting an enhancement pixel for the target pixel. 

[12] By allowing a partial match for enhancement instead of requiring the 

10 100-percent match of the prior art, such a method provides image-resolution 

enhancement for digital-image data that may be mildly corrupted by noise or other 
errors. 

Brief Description of the Drawings 

[13] The invention, together with objects and advantages thereof, may best 

15 be understood by making reference to the following non-limiting description taken in 
conjunction with the accompanying drawings, in the several figures of which like 
referenced numerals identify like elements, and wherein: 

[14] FIG. 1 illustrates a conventional 5X5 window pixels located about a 

target pixel; 

20 [15] FIG. 2A illustrates the pixel values of a 5 X 5 window of an image; 

[16] FIG. 2B illustrates a mask portion of a template that establishes 

whether a particular pixel in the 5 X 5 window of FIG. 2A is a pixel of interest; 

[17] FIG. 2C illustrates a pattern portion of a template that defines a 

"prediction-bit subset" that establishes the value each particular pixel of interest must 
25 have to be considered a match; 

[18] FIG. 2D illustrates a match being found in the prior art when all of the 

pixels of interest (the pixel-bit subset of the window as defined by the mask of FIG. 
2B) in the 5X5 window of FIG. 2A each have the required value as defined by the 
pattern (the prediction-bit subset) of FIG 2C, 
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[19] FIG. 3A illustrates values and position of pixels in a 5 X 5 window of an 

image similar to the window of FIG. 2A with an error in one of the pixel values 
according to an embodiment of the invention; 

[20] FIG. 3B illustrates a mask portion of a template that establishes 

5 whether a particular pixel in the 5X5 window of FIG. 3A is a pixel of interest, 
according to an embodiment of the invention; 

[21] FIG. 3C illustrates a pattern portion of a template that defines a 

"prediction-bit subset" that establishes the value each particular pixel of interest must 
have to be considered a match, according to an embodiment of the invention; 

10 [22] FIG. 3D illustrates an "N" representing a "no-match" status at pixel 5D 

of the 5X5 window of FIG. 3A because of the error in binary data, and an absence 
of a 100-percent match, according to an embodiment of the invention. 

[23] FIG. 4 is a flow chart of a method of image enhancement employing 

partial template matching, according to an embodiment of the invention; and 

15 [24] FIG. 5 is a diagram of a resolution-enhancing device, according to an 

embodiment of the invention. 

Detailed Description 

[25] FIGS. 3A-D illustrate a method of image enhancement employing 

partial-template matching, according to an embodiment of the invention. The 

20 window of FIG. 3A represents a scan of a 5 X 5 portion of an image similar to the 5 X 
5 image portion represented in the window of FIG. 2A, except that the scan was not 
perfect and introduced a data error at pixel 5D in the form of a "1" instead of a "0." 
The template represented by the mask of FIG. 3B and the pattern of FIG. 3C are the 
same as FIGS. 2B and 2C. Applying the mask and pattern of the template of FIGS. 

25 3B and 3C to the window will not produce a 1 00-percent match because of the data 
error. FIG. 3D illustrates an "N" at pixel 5D representing a "no match" status. Under 
the prior art, because no match was found because the binary data was not perfect, 
no enhancement pixel is substituted for the target pixel 3C. Thus, as discussed 
above in conjunction with FIGS. 1-2D, the prior art leaves the image represented by 

30 the window of FIG. 3A without resolution enhancement. 
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[26] One embodiment of the invention modifies the prior-art-resolution- 

enhancement technology to require only a partial match be found rather than a 100- 
percent match between the pixels of the window and the pattern for an enhancement 
pixel to be substituted for the target pixel. The degree of partial match may be 
5 selected to provide a desired degree of image-enhancement accuracy. For 

example, a partial match may be found if at least 90 percent of the bits of the pixel- 
bit subset of the window as defined by the mask of FIG. 3B equal the pixels of the 
prediction-bit subset as defined by the pattern of FIG 3C. In an alternative 
embodiment, a partial match may be found if at least 80 percent of the bits of the 
10 pixel-bit subset compare with the prediction-bit subset. 

[27] For the window illustrated in FIG- 3A, a match for the purposes of 

enhancing the pixel 3C is found because 13/14=93 percent of the pixels of interest in 
the window of FIG 3A match the pattern of FIG 3C. Therefore, the pixel 3C is 
enhanced, for example, by darkening only its upper-left quadrant. 

15 [28] In a further embodiment of the invention, the comparing step can 

include assigning weights to each matched bit, and each matched bit is multiplied by 
the corresponding weight prior to determining if there is a match. For example, 
pixels further from the target pixel 3C could be weighted less. 

[29] FIG. 4 shows a flow chart 20 of a method of image enhancement 

20 employing partial-template matching according to an embodiment of the invention. 

[30] After a start, the flow begins at block 22 where the image for resolution 

enhancement is received, usually as pixel values in binary form. 

[31] Next, at block 24, at least a portion of the image for enhancement is 

stored in a memory or buffer module in a two-dimensional array of lines and 
25 columns. 

[32] The logical flow moves to block 26 where a target pixel from the image 

stored in memory is selected. 

[33] Proceeding to block 28, a window is selected comprising a plurality of 

line segments of bits representing pixels located about the target pixel, the window 

30 including pixels from the same line as the target pixel as well as pixels from at least 

one line above and at least one line below the target pixel, if available. The size and 
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shape of the window depends on the image to be enhanced, and the particular 
enhancement process to be performed on the image. 

[34] The flow moves next to block 32 where the window comprising the 

plurality of line segments of bits representing pixels is compared with at least one 
5 template of the set of templates 30. The comparison may be performed in hardware, 
firmware, software, or a combination thereof. The comparison is performed as 
described in conjunction with FIGS. 3A-D. 

[35] The flow moves to decision block 34 where a determination is made 

whether a match was found. The degree of match may be determined as described 
10 in conjunction with FIG. 3. 

[36] If a partial match is found, the flow moves to block 36 where an 

enhancement pixel is substituted for the target pixel. 

[37] After block 36, the flow moves to decision block 38 to determine 

whether all the pixels of the image have been subjected to the resolution- 
15 enhancement process. If yes, the flow moves to the end block. If no, the flow 

moves back to block 26 and selects another target pixel for resolution enhancement. 

[38] If a partial match is not found at decision block 34, the flow moves to 

block 38. Upon completion of the flow of the resolution-enhancement method 20, all 
the pixels comprising the image have been processed and an enhancement pixel 
20 has been substituted in each instance where a partial match occurred. 

[39] One embodiment of the invention uses hardware including logic gates 

to implement the resolution-enhancement method 20. The hardware is operable at 
block 32 to compare a plurality of templates of the set of templates 30 to the selected 
window. Typically, the hardware would compare all of the templates of the set of 
25 templates 30 to the window at the same time. Multiple template matches may occur 
when simultaneously comparing templates. A priority scheme may be established 
for selecting which template constitutes the partial match and determines the 
enhancement pixel. 

[40] Another embodiment of the invention uses firmware and/or software to 

30 implement the resolution-enhancement method 20. In this embodiment, templates in 

the set of templates 30 are each assigned a priority. At the template-compare block 
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32, the templates are compared in order of priority, beginning with the highest 
priority. The first template that partially matches wins and determines the 
enhancement pixel. 

[41] FIG. 5 illustrates a diagram of a resolution-enhancing device 50 

5 according to an embodiment of the invention. The resolution-enhancing device 50 
includes a memory 52, a selector module 54, templates 56, a logic module 58, and a 
pixel-substitution module 60. The resolution-enhancing device 50 typically is 
incorporated into the rendering device, such as a printer or computer, or may stand- 
alone. Further, the elements of the resolution device may be distributed among 
1 0 several devices. For example, the memory 52 may be a part of a personal computer 
and the remainder of the device 50 may be a part of a printer. 

[42] The memory 52 can be any type of memory operable to store 

consecutive lines of an image being enhanced, and preferably is a fast 
semiconductor storage device. The selector module 54 may be any type of device 

15 operative to select from the image a window comprising a plurality of line segments 
of bits representing pixels, the window including a target pixel. The selector module 
54 is operative to select a target pixel and select the window about the target pixel, 
as generally described in the prior art. The selector module 54 is further operative to 
sequentially select pixels in the image as a target pixel until all the pixels in the 

20 image have been selected. The templates 56 include a plurality of templates, as 

generally described in the prior art. Each template includes a mask, as illustrated in 
FIG. 2B, that establishes whether a particular pixel in a window is a pixel of interest, 
and a pattern illustrated in FIG. 2C that defines a "prediction-bit subset" that 
establishes the value each particular pixel of interest must have to be considered a 

25 match. The logic module 58 is operative to compare the pixels of the window with 
the templates 56, also as generally described in the prior art. However, and unlike 
the prior art, the logic module 58 is operative to find a partial match between the 
pixels of the window and a template of the templates 56. If a partial match is found 
with a plurality of templates, the logic module 58 may be further operative to apply a 

30 priority scheme to determine which template is used to substitute the enhanced pixel 
in response to the partial match. The pixel substitution module 60 is operative to 
substitute an enhancement pixel for the target pixel responsive to a partial match 
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being found in the logic 58. The pixel-substitution module 60 may be a portion of the 
logic 58, but is illustrated here as a separate module for description of the function of 
substituting an enhancement pixel for the target pixel responsive to a partial match 
being found. 

5 [43] The resolution-enhancing device 50 typically is interposed between a 

source of an image to be rendered and the rendering device 76, the source typically 
being in pixel form. The resolution-enhancing device 50 receives the input image 72, 
performs the method 20 of image enhancement described in conjunction with FIG. 4, 
and provides an enhanced-image output 74 to a rendering device 76. The rendering 
10 device 76 typically includes a printer, but may be a selected one of a monitor, a 
projection unit, or a plotter. In other embodiments, partial template-matching 
functions may be implemented with firmware and/or with software on the user's host 
computer. 

[44] An embodiment of the invention can be implemented using an 

15 application specific integrated circuit (ASIC). The templates are stored and the 
matching occurs in the ASIC gates. The templates are described in hardware 
descriptive language and the code synthesized into a portion of the logic of the 
ASIC. In an alternative embodiment, some newer ASIC video blocks have several 
programmable table entries in addition to the fixed tables in hardware. These 
20 programmable table entries can be loaded by the printer's firmware from external 
ROMs into the ASIC tables. The tables are read directly by the hardware and the 
comparison is done at the same time as the built-in table matches. 

[45] The various embodiments of the invention may be implemented as 

interconnected machine-logic circuits or circuit modules and/or as a sequence of 

25 computer implemented steps or program modules running on a computing system. 
The implementation is a matter of choice dependent on the performance 
requirements of the system implementing the invention. In light of this disclosure, it 
will be recognized by one skilled in the art that the functions and operation of the 
various embodiments disclosed may be implemented in software, in firmware, in 

30 special purpose digital logic, or any combination thereof without deviating from the 
spirit or scope of the present invention. 
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[46] Although the present invention has been described in considerable 

detail with reference to certain preferred embodiments, other embodiments are 
possible. Therefore, the spirit or scope of the appended claims should not be limited 
to the description of the embodiments contained herein. It is intended that the 
5 invention resides in the claims hereinafter appended. 
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